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Abstract: This paper focuses on the design, implementation and evaluation of an online game for elementary and middle 
school mathematics. Its aim is twofold: (a) the development of the prototype of a flexible and adaptable computer game, 
and (b) the evaluation of this prototype, as to its usability and technical aspects. The particular computer game was 
created in an attempt to facilitate the teaching of mathematics, a subject that is often regarded as complicated by students 
of all ages. Apart from the game, an administration website was also constructed, so that the educator can configure the 
game, without that requiring any programming skills. More specifically, the educator can use the administration website in 
order to alter several of the game's parameters, such as the content and total number of its questions.The game was 
evaluated in real school settings, both through a pilot study with 12 students and a longterm intervention with 37 students 
that lasted 14 weeks. The results indicated that the students' opinion about the game was positive, and suggest that with 
some extensions the game could be used as an effective learning tool. Finally, some corresponding conclusions and future 
improvements to the game are being discussed on the basis of the findings. 
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1. Introduction 

Research into the education of mathematics has long demonstrated that math learning difficulties is a 
common and important problem among students of all ages. According to Garnett (1998), many students face 
math learning problems of different types; these learning difficulties range from mild to severe, and require 
instructional attention and various treatment methods. Some of the most common math learning problems 
include: (a) difficulty memorizing basic number facts; (b) computational and arithmetic weakness; (c) confusion 
about terminology and the written symbolic notation system of school math; and (d) weak understanding of 
concepts due to visual-spatial organization deficits (Garnett, 1998). Apart from lower performance in math 
exercises and tests, these math learning disabilities can also result in avoidance behavior and negative 
perception of the particular subject. Often, students with math learning difficulties exhibit high math anxiety, 
which is defined as "a feeling of tension, apprehension, or fear that interferes with math performance" 
(Ashcraft, 2002). This math related problem was first reported by Dreger and Aiken (1957), who noticed that 
students demonstrated emotional reactions to arithmetic and mathematics. According to their study, math 
anxiety is distinct from general anxiety, not necessarily related to overall intelligence, and can contribute to 
poor performance in mathematics. These findings were substantiated and reinforced by more recent research 
(Richardson and Suinn, 1972; Tobias and Weissbroad, 1980; Wingfield and Meece, 1988; Ashcraft and Kirk, 
2001), indicating that there is a negative correlation between math achievement and math anxiety. 

One of the most detrimental consequences of math anxiety is that it can lead to unfavorable attitude towards 
the particular subject, as well as negative self-perceptions about one's math abilities (Ashcraft, 2002). 
Consequently, highly math-anxious students also tend to avoid enrollment in math-related courses, as well as 
pursuing degrees or career paths based on mathematical or quantitative skills (Hembree, 1990; LeFevre, Kulak 
and Heymans, 1992). Hence, the educator should try to incorporate teaching methods that emphasize the 
value of mathematics, help students develop their math skills, and increase their self-efficacy beliefs (Meece, 
Eccles and Wingfield, 1990). Moreover, it is of utmost importance to help students acquire a positive 
perception of mathematics, as this is considered to be highly related to lower math anxiety and higher math 

ISSN 1479-4403 230 ©ACPIL 

Reference this paper as: Aikaterini Katmada, Apostolos Mavridis and Thrasyvoulos Tsiatsos "Implementing a Game for 
Supporting Learning in Mathematicss" The Electronic Journal of e-Learning Volume 12 Issue 3 2014, (230-242) available 
online at www.eiel.org 







Electronic Journal of e-Learning Volume 12 Issue 3 2014 


achievement (Hembree, 1990). This could be achieved though the use of computer games, since they 
encompass many characteristics that make them valuable tools for the educational process. More specifically, 
computer games promote active learning (Oblinger, 2004) and the development of various skills (McFarlane, 
Sparrowhawk and Heald, 2002), while they retain their entertainment and appealing qualities (Kafai, 2001). 

So far, effective use of computer games for educational purposes has been reported in various subject areas, 
such as geography (Virvou, Katsionis and Manos, 2005; Tuzun et al., 2008), computer science (Papastergiou, 
2008), health education (Dorman, 1997), and mathematics and sciences (Klawe 1999; Annetta et al., 2009). 
According to Ke (2009), who conducted a meta-analysis with 89 empirical studies on instructional gaming, 
computer games can affect favorably students' motivation and learning in a multitude of educational settings, 
both formal and informal. Additionally, as Hays (2005) points out, specially designed instructional computer 
games can be of educational value. Nevertheless, it should be noted that the majority of the existing empirical 
studies are inconsistent due to divergent research, game, and learner variables, short-term experiments and 
interventions, and unclear descriptions of the games that were used (Ke, 2009). 

Accordingly, this study addresses the design and development of a computer game that could be utilized as an 
adaptable tool for the educational process. The purpose of the particular game would be to support the 
teaching of elementary and middle school mathematics, as a complementary learning tool that could enhance 
students' motivation and engagement with the subject. Apart from the game, an administration website was 
also constructed, so that the educator can ensure that the game's content aligns with the curriculum and the 
learning goals of the current lesson. Moreover, the educator can easily edit the game's content and images, or 
upload new ones, without that requiring any programming or scripting skills. Concerning the game's design, 
basic educational computer game design principles (Malone, 1980; Prensky, 2001; Hays, 2005; Fisch, 2005) 
were taken into account, so that the game would be educationally and motivationally effective. The game 
prototype was then evaluated both through a pilot study and a longterm (14 weeks) intervention, in order to 
detect any technical flaws, and to assess its usability and educational aspects, so that it could be revised and 
improved in subsequent editions. 

Hence, this paper aims at presenting a concrete case study on the design and development of the game 
'Volcanic Riddles', as well as the results of a more longitudinal evaluation study in real school settings, where 
the aforementioned game was used for the teaching of formal curricular material. In order to investigate the 
criteria mentioned above, the research questions of the particular study are the following: 

RQ1: What is the students' opinion regarding the usability of the 'Volcanic Riddles' game? 

RQ2: In what educational context could this game be used by educators? 

RQ3: Will the gender or grade of students affect their opinion about the usability of the'Volcanic 

Riddles' game? 

This paper is organized as follows: in the next section, the conceptual framework that inspired the design of 
the aforementioned game is being reported. More specifically, Digital Game-Based Learning, as well as several 
studies concerning the use of computer games in mathematics education are being thoroughly analyzed. Next, 
there is a presentation of the aforementioned game, and the principles that inspired its design. Afterwards, 
the basic features of a prototype of this game and the administration website are being presented, and, 
subsequently, there is a description of the evaluation studies of this game's prototype. Finally, the results of 
the studies are being discussed, and some conclusions are drawn on the basis of the findings. 
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2. Conceptual framework 

2.1 Young people and Digital Game-Based Learning 

Recently, there has been an ongoing interest in the use of computer games for educational purposes, as a 
means to increase students' motivation, engagement and achievement in various subject areas. 

This learning approach, which combines digital game-based activities and educational content, is often refered 
to as Digital Game-Based Learning (DGBL). DGBL is considered able to render the learning of difficult or 
uninteresting subjects more accessible, engaging, and enjoyable (Malone, 1980; Kafai, 2001). Indeed, 
computer games play an integral role in today's children's lives, being part of their culture and one of their 
most frequent and favored activities (Mumtaz, 2001; Fromme, 2003). The reason for that could be their 
intrinsically motivational appeal and the fascination they provoke to young people, by encompassing elements 
of curiosity, challenge, and fantasy (Malone, 1980). Moreover, it has been suggested that computer games 
have the ability to immerse players in a state of 'flow' (Csikszentmihalyi, 1990), characterized by deep and full 
involvement and enjoyment in the activity. This state of 'flow' was described by Csikszentmihalyi (1990: 4) as 
"the state in which people are so involved in an activity that nothing else seems to matter". Additionally, 
scholars have argued that computer games could be more effective and more appropriate than traditional 
instructional methods for the current generation of learners, whose cognitive abilities and interests are 
influenced by digital games and technology (Facer, 2003; Srinivasan, Butler-Purry and Pedersen, 2008). 

Given the benefits described above, DGBL has already been implemented in various sectors, with three 
different approaches that, according to Van Eck (2006), are the following: (a) students create their own 
educational games, with the aid of the educators, (b) educators use commercial games in class that have not 
been primarily developed for educational purposes, (c) educators use games in class specifically designed for 
education by other educators, instructional designers, and developers. These instructional games, also known 
as 'serious games', are usually designed for training purposes and have many applications in various fields, 
such as education, science, production, and health (Sawyer and Smith, 2008). Each of the three approaches to 
implementing DGBL has advantages of its own; however, the most prominent benefits that DGBL offers, in 
general, can be summarized as follows: (a) computer games can facilitate the acquisition and transfer of 
knowledge to new situations through feedback and self-assessment mechanisms (Oblinger, 2004); (b) they can 
also promote the development of problem-solving and memorization skills (McFarlane, Sparrowhawk and 
Heald, 2002); (c) they help students familiarize themselves with technology, as well as programming and 
computer science concepts (Kafai, 2001; Van Eck, 2006; Prensky, 2008); and (d) gaming is often a social 
activity, contributing to the development of the players' social and emotional skills (Squire, 2003; Fromme, 
2003; Oblinger, 2004). 

Nevertheless, DGBL constitutes a relatively recent and still evolving instructional method, and there is a need 
for more empirical evidence that could validate its actual educational value, and show how it could be applied 
more effectively. As Flays (2005) points out, research has demonstrated that games can promote learning in 
various subject areas, however there is no proof that games can be used in all situations and for every 
instructional task. It is also important that students are provided with debriefing, feedback, and support from 
the educators during the DGBL activities (Flays, 2005). Moreover, there are many practical issues that may 
deter the educator from using computer games in the classroom. A common impediment is that many times 
the game's content is not correct or it does not align with the curriculum and the learning objectives of the 
classroom (Kirriemuir and McFarlane, 2003; Fisch, 2005). Also, teachers often encounter difficulties in 
identifying the educational components of a game, as well as integrating the game in the traditional 
educational process (Kirriemuir and McFarlane, 2003; Baek, 2008). Additionally, many contemporary complex 
games require new hardware and plenty of time, and thus they cannot be played in the classroom (Kirriemuir 
and McFarlane, 2003). Lastly, a frequent concern of educators and parents alike is the possible negative effects 
of gaming on children, such as addiction and overly competitive behavior (Baek, 2008). 
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2.2 Review of research on computer games for mathematics 

There are several studies that report on the use of commercial computer games for mathematics, or present 
the development and evaluation of instructional games designed for the specific subject. As indicated by the 
following review of relevant studies, computer games can increase students' math achievement and 
performance, and promote positive attitudes towards mathematics. For instance, in a recent study, Pareto et 
al. (2011), created a teachable-agent arithmetic game that aims in training basic arithmetics skills. The game 
was evaluated in a study with 153 participants, consisting of 3 rd and 5 th grade students. The results indicate 
that the game helped students improve their math performance and self-efficacy beliefs. Ahmad and Latih 
(2010) describe the development of an educational math game on fractions for primary school students. 
Similarly, Lee (2009) report on the creation and evaluation of an education game on fractions and mention 
that it improved students' understanding and performance. 

Concerning the use of commercial games for mathematics, Zavaleta et al. (2005) suggest in their study that the 
use of a commercial game for elementary school algebra enhanced students' achievement. Kebritchi, Hirumi, 
and Bai (2010) investigated the impact of commercial math games on 193 high school students' math 
performance, with positive results concerning the student's perception of mathematics, motivation, and 
achievement. Ke and Grabowski (2007) examined the effects of the use of adventure games on 125 5 th grade 
students that were assigned to three groups: cooperative game playing group, competitive game playing 
group, and no game playing group. According to their findings, after the four-week intervention the two game 
playing groups had better math performance, while the cooperative game playing group had better attitude 
towards the subject, compared to the other conditions. In another study, Ke (2008) examined the effects of 
the use of computer games on 4 th and 5 th grade students that were enrolled in a five-week summer math 
program, with positive results concerning the students' attitude towards math. More recently, Kim and Chang 
(2010) performed regression analyses using 170,000 4 th grade students, demonstrating that math computer 
games had a positive effect on male minority students. 

Other studies focus on the use of math computer games for the remediation of specific deficits, such as 
dyscalculia. For example, Wilson et al. (2006) created an adaptive computer game for dyscalculia and tested it 
in a five-week evaluation study with nine children with math learning difficulties. The results indicated an 
increase in the children's math performance on core number sense tasks, as well as an improvement as 
regards their confidence in their mathematical abilities. Regarding any pertinent research projects, the E-GEMS 
('Electronic Games for Education in Math and Science', http://www.cs.ubc.ca/nest/egems/ ) project 
contributed on the development of various educational games that increased student engagement and 
achievement, and produced several design heuristics (Klawe & Phillips, 1995). 

3. Design of the game 

3.1 Basic features of educational games 

According to Malone (1980), instructional designers should try to create intrinsically motivating educational 
environments that would help students learn in an effortless and engaging way. Computer games, in 
particular, contain the following three elements that make them so interesting, and can be used in order to 
motivate the learner: challenge, fantasy, and curiosity (Malone, 1980). Malone (1980) draws upon these 
observations in order to develop a set of guidelines for the design of effective and motivational instructional 
computer games. In accordance with these guidelines, games should have clear goals, uncertain outcomes, 
feedback, and gradually increasing difficulty levels. Furthermore, they should contain curiosity and fantasy 
elements, such as emotional aspects. Moreover, they should respond in an appropriate way to the players' 
actions, and they should provide them with choice over various environmental aspects (Malone, 1980). 
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Hays (2005) reports on these heuristics, and suggests a set of design recommendations for educational games, 
emphasizing on the instructional quality these games should have. More specifically, the game should be 
integrated into a larger educational program, and it should also incorporate elements that help students build 
new knowledge structures or complete their existing ones (Hays, 2005). Furthermore, as stated by Fisch 
(2005), the appropriate incorporation of educational content into the game is a key factor in the design of 
effective instructional games. Students should also be provided with offline material and resources that could 
add to the game's educational value, as well as support and guidance by teachers and parents (Fisch, 2005). 
Prensky (2001), points out some features that engaging games have; these are the following: objectives, 
opposition, interaction, representation, and outcomes. Similarly, Kiili (2005) proposes an experiential gaming 
model based on Csikszentmihalyi's flow theory and experiential learning principles; this model can be used for 
the design, analysis and evaluation of educational computer games. The above design guidelines and basic 
features of instructional games were taken into consideration in the design of the proposed game. 

3.2 Development of the game 

The final framework consists of the configurable online 2D game and its administration website, which was 
constructed in order to facilitate the (non-programmer) educator in the configuration of the game's 
parameters. In accordance with the ADDIE model for Instructional Systems Design (ISD), the production of the 
framework comprised the following working phases: Analysis, Design, Development, Implementation, and 
Evaluation. Hence, a careful and thorough requirements analysis was deemed necessary, in order to determine 
the conditions that the particular instructional game should meet. The researchers were allowed to observe 
the educational process and the students' performance during classes in a week's time. Moreover, a semi- 
structured interview with two elementary school teachers was conducted, as well as a carefull review of the 
relevant literature. 

After careful consideration of the teachers' recommendations, it was decided that they game should be 
simple, without any distracting material, and that it should not require any software installation. This way, it 
would be easier for teachers to use it in formal school settings. Concerning the difficulty level of the game's 
questions, the educators suggested that it should increase as the game progresses. In addition, they stated 
that they would like to be able to change the game's questions, according to the student's knowledge and 
skills. Hence, a very important objective of this work was to create an adaptable game that could be reused in 
various educational settings and activities. It was also decided that the game should contain various mini 
challenges that the students could play in the limited time span of an individual class. That way, the game 
could be tailored to the level of the students, and it could also support different thematic units of the subject. 

Afterwards, the game and its administration website were developed. More specifically, the game's graphics 
and images were designed in Adobe Photoshop and Adobe Illustrator, and then the researchers proceeded 
with the development of the game's prototype. Regarding the game development software, the game was 
created in Adobe Flash with the use of ActionScript, while the administration website was created using the 
PHP scripting language. The changeable content is saved in text files that were uploaded to the web server, 
and can be edited via the administration website. In accordance with the ADDIE model, the product of each 
working phase was subjected to formative evaluation and revisions. This way, it was confirmed that the initial 
criteria that had been set for the final product were met, and that any technical flaws were detected and 
corrected in time. Thus, the proper functioning of both the game and its administration website was ensured. 

4. Description of the game 

The game consists of nine challenges that the educator can fill up with questions, according to the course 
material. Since the game is addressed to elementary and secondary school students, attention was paid so 
that it is friendly and easy to use. An attempt was also made to incorporate most of the aforementioned 
characteristics of educational games into the proposed game. In this section, the main features of the game 
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are being presented, in connection with the proportionate characteristics that inspired the game design. 
Afterwards, there is a brief description of the configuration of the game through the administration website. 

4.1 Story and characters 

In accordance with Malone's design heuristics (1980), one of the features that make games intrinsically 
motivating is the theme or fantasies that are incorporated. These two elements can elicit mental images of 
social situations or of physical objects to the players, rendering the game more interesting and engaging 
(Malone, 1980). Moreover, apart from being a source of motivation, the game's narrative can be considered as 
a supportive factor that helps players make sense of the information they are presented with. Additionally, it 
can help students learn more efficiently, especially in case of difficult content (Waraich, 2004). For that reason, 
a theme and a story were added to the game. In the game's story, three fictional friends, a boy, a girl, and a 
robot are vacationing on an exotic island, when the children's uncle sends them a letter to warn them that the 
island's volcano is going to erupt. The three friends face nine challenges in order to gather the necessary 
supplies and find a way to escape from the island. The player has to help them complete these challenges 
successfully. Each one of these challenges contains a variable number of questions on various algebraic and 
geometrical concepts. Special attention was paid so that the questions are well integrated into the game's 
story. For example, in one instance of the game, the three friends visit the island's store in order to buy 
supplies; the players have to buy the products with the most favorable prices after the discount, using their 
knowledge on solving equations with percentages. The players can read the game's story and information 
about the three heroes of the game via the 'Options' menu of the game. Each of the game's challenges is 
connected to the storyline of the game through its objectives and narrative. That way, the players can choose 
one challenge to play and do not need to play the whole game every time. 

4.2 Goals and rules 

Another important characteristic of compelling instructional games that was integrated into the particular 
game is the presence of clear rules and goals. More specifically, the players' can choose from two different 
game modes: "Challenge game" and "Single game" mode. When playing in Challenge game mode, the players 
have to successfully complete the nine challenges of the game, while in Single game mode they can choose a 
challenge to play. The players have a one hundred seconds for each question, as well as five available lives 
(chances) for each challenge. They lose one life for every mistake they make. According to the rules of the 
game, when the players play in Single game mode and they lose, they can restart the same challenge or start 
another one. Otherwise, if they were playing in Challenge game mode, they have to start from the first 
challenge again. 

4.3 Opposition elements 

Furthermore, the game contains several elements of challenge and opposition, which, according to Prensky 
(2001), are problems the players are trying to solve. These elements make the game more engaging, however 
they should be equivalent to the player's abilities (Prensky, 2001). In order to win the players have to complete 
successfully all the challenges of the game. Flowever, the difficulty of the questions increases gradually, as the 
challenge progresses. Moreover, in the fourth challenge a different approach was followed; here, the difficulty 
of the questions increases with each correct answer, and decreases with each false one. Hence, the level of 
difficulty adapts to the player's skills and level. As it has already been mentioned, the players have five lives for 
each of the game's challenges at their disposal, and they lose a life each time they make a mistake. When they 
lose all of their lives, the game ends. Moreover, in accordance with Malone (1980), a score-keeping 
mechanism and a countdown timer were added, in order to create goals of different levels. The players have 
one hundred seconds to answer each question otherwise they lose. They also win points for each correct 
answer and lose points for each false one; answers to more difficult questions are rewarded with more points. 


www.ejel.org 


235 


©ACPIL 



Aikaterini Katmada, Apostolos Mavridis and Thrasyvoulos Tsiatsos 


Each player's high-score is saved and can be accessed through the 'Options' menu, so the players can compare 
their high-scores. 

4.4 Interaction and feedback 

According to Prensky (2001), engaging educational games should also contain interaction and feedback 
mechanisms that will help the player learn. Interaction can be either between the player and the computer, or 
between players (Prensky, 2001). The proposed game provides the players with immediate feedback 
messages, in order to help them understand and correct their mistakes. Moreover, directions and help 
messages appear in each of the game's challenges, so that they players understand how the game works and 
learn to navigate through it. The players can also view their high-scores, and configure some of the game's 
parameters, such as the game's sounds, through the "Options" menu. Additionally, they can communicate 
with the authors and creators of the game via a web form, in order to send their suggestions and comments 
(Figure 2). The players can interact with various elements of the game; however, the game did not support 
interaction between the players by the time it was evaluated. 



Figure 2: Various screens of the game. 

4.5 Configuration options 

As it has already been mentioned, one of the main purposes of this work was the development of an easily 
configurable online game. The whole framework that was created consists of the online game and its 
administration website. Using this administration website, the educator can change the following: (a) the total 
number of each challenge's questions, (b) the content of each question and its corresponding answer, (c) the 
instructions and help messages for each challenge, and (d) the images that appear in the game, as well as the 
images of the help messages. Moreover, in the fourth challenge the educator can add a set of more difficult 
questions that will be used when the player answers correctly, in order to increase the level of the questions' 
difficulty. The configuration process is the following: the educators should login to the administration website 
and then select and edit the game's messages, directions, questions and answers. They can also delete 
questions and answers, or add new ones, and furthermore, they can upload new images or delete the existing 
ones (Figure 3). Finally, they should save their changes in order for them to appear in the game. 
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Figure 3: The administration website of the game. 

5. Evaluation of the game's prototype 

5.1 Description of the evaluation activities 

An important objective of this study was the evaluation of the game as to its usability, capabilities and 
perceived usefulness, in order to improve it in subsequent editions. For that reason, several educational 
activities, which served as a means to gather qualitative data, were organised and implemented. Hence, the 
researchers had the opportunity to observe the students' reaction to the game, and to ask them for their 
feedback. Additionally, in order to gather quantitative data, a pilot study with twelve (12) 6 th grade students of 
a private school was conducted. The group of students consisted of eight (8) girls and four (4) boys of various 
nationalities, aged 10-12 years old. 

The pilot study comprised two different educational activities conducted in the computer laboratory of the 
school. Each one lasted about 45 two differThe questionnaire for this study included 18 questions focused on 
the gamein the computer laboratory of the school) boys of various nationalities, aged 10-12 years old. tions. 
For that reasonere 1 was assigned to 'strongly disagree' and 5 to 'strongly agree'. Regarding the game's 
content, each challenge was enriched with new questions on arithmetic and geometrical concepts, in 
accordance with the material taught in the classroom. The evaluation data also included the researchers' notes 
and observations from the educational activities. The results were quite encouraging; however, they could not 
be generalized, due to the limited number of the participants and that of the educational activities. 
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For that reason, a second evaluation experiment that lasted 14 weeks was also conducted. The participants of 
this study were 37 randomly selected students from a typical public secondary school, consisting of 23 boys 
and 14 girls aged 12-14 years old. The students agreed voluntarily to participate in the experiment by 
interacting with the game daily and individually. Responsible for supporting them during the whole 
intervention was primarily their teacher. Before the intervention could begin, the cooperating teacher 
organised an introductory session, where the researchers presented the game to the students. Moreover, the 
teacher informed the students' parents about the intervention and they signed the necessary parent consent 
forms. 

For the purpose of this evaluation a new, more complete, paper-based questionnaire was constructed. The 
particular questionnaire, which elicited both qualitative and quantitative data, was based on the Questionnaire 
for User Interface Satisfaction (QUIS) (Chin, Diehl and Norman, 1988), and on Lund's USE Questionnaire (USE) 
(2001). The 22 questions that were included in the questionnaire corresponded to the following eight 
dimensions: perceived usefulness, ease of use, ease of learning, satisfaction, screen, terminology and system 
information, system capabilities, and overall reaction to the software. The questions' type was five-point Likert 
scale, as before. Additionally, the questionnaire included two open-ended questions about the best and worst 
aspects of the game. 

The participants played the game from home daily during the 14 weeks of the intervention. The teacher was 
able to check their involvement, as all participants had personal accounts. Every week, the teacher updated 
the game's challenges with new questions based on what was taught in the lessons. Moreover, every two 
weeks a debriefing session was conducted in the school computer laboratory. At the end of the intervention, 
the students completed the anonymous questionnaire. In the next section, the results of the analysis of the 
questionnaire are being analytically presented. 

5.2 Results 

The students' answers to the Likert type questions of the questionnaire were analyzed by descriptive statistics, 
and their answers to the open-ended questions were grouped according to their common themes. The 
statistical analyses were performed using the SPSS 20 statistical package, with the level of significance set to 
0.05. The game's usability was assessed using the students' performance in each dimension. This was 
calculated using the mean score of their answers to the corresponding questions (Boone and Boone, 2012). 
Generally, as the results indicated, the students' opinion about the game's usability was moderate to good. In 
particular, regarding the gamedinease of learning, the statistical results are the following: M = 3.77, SD = 1.07, 
N = 37. The studentsg, the staticoncerning the gametatistical resuls terminology and system information are 
also positive, as the statistical results are M = 3.67, SD = 0.84, N = 37, and M = 3.63, SD = 0.77, N = 37, 
respectively. Furthermore, their responses to the rest of the questionnaire's dimensions were all above 
average, while the mean score for the students score est of the questionnaire= 37, and M = 3.63, SD = 
0.770.96, N = 37). However, the lowest score was observed in the dimension concerning the system 
capabilities (M = 3.33, SD = 0.98, N =37). 

In addition, according to the students' answers to the open-ended questions, the game: (a) helped them 
understand the lesson, as well as some difficult mathematical concepts better; (b) it is easy, entertaining and 
pleasant; (c) it helps apply a more innovative approach to the learning process; and (d) it is flexible and, thus, it 
can constitute a useful tool for the revision of the lesson taught. Thus, concerning the first research question 
(What is the students' opinion regarding the usability of the 'Volcanic Riddles' game?), it was deduced that the 
students were quite positive. 

However, some students encountered minor problems regarding the use of the game. For example, five 
students stated in their comments that they would prefer the game to be in the Greek language, instead of 
English. Other observations were that it is quite simple, and it does not provide enough explanations on some 
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solutions. Their opinion can also be justified by the fact that they mostly played the game from home, without 
the aid of their teacher. It should be noted that the studentld responses to the related questions during the 
pilot study were a lot more positive. For instance, regarding the game were a lot more positive, pilot studyd 
that it was very easy to understand how the game is played, while two pupils were neutral (M=4.67, SD=0.778, 
N=12). Also, seven pupils strongly agreed and three pupils agreed that the game did not require any complex 
computer usage, whereas two pupils were neutral (M=4.42, SD=0.793, N=12). Eight pupils strongly agreed and 
three pupils agreed that the game could help them improve their skills, whereas one pupil was neutral 
(M=4.58, SD= 0.669, N=12). The students game were a lot more positive, pilot studyducture, functions, and 
motivational appeal were also quite positive. However, only three students strongly agreed that they use the 
game's "Help" option, whereas two students agreed and seven students strongly disagreed (M=2.50, SD=1.88, 
N=12). Indeed, the students preferred to ask the teacher and the researchers when they did not know or 
understand something in the game. 

These results can be explained on the premise that the pilot study included a small number of activities (2), 
while in the longterm intervention the students had 14 weeks to explore the game. Moreover, in the pilot 
study, the activities took place during class with the assistance of the teacher, and in the form of a pleasant 
break, while in the longterm experiment the students would log in from their homes and in their free time, in 
order to play the game. Lastly, concerning both teachers 1 opinion about the game and its administration 
website, it was quite positive, as they considered them to be friendly and easy-to-use. Hence, regarding the 
second research question [In what educational context could this game be used by educators?), it was 
concluded that the game could actually be used as a useful educational tool for classroom activities. Moreover, 
with some improvements and extensions, it could also be used in distance-learning scenarios. Additionally, it 
became apparent that, in accordance with Hays (2005), the educator's feedback and the debriefing sessions 
are very important for the support and guidance of the students. 

Additionally, a third research question (Will the gender or grade of students affect their opinion about the 
usability of the 'Volcanic Riddles' game?) was also investigated in this evaluation study. Firstly, it was examined 
whether the students' opinion for the game differs based on gender. After some preliminary tests were 
conducted on the data, it was decided to use the Mann-Whitney U Test, since it can be used when the 
variables are not necessarily normally distributed. According to the results, it can be presumed that there were 
no significant differences between boys' and girls' opinions regarding the game, based on the scores for the 
eight dimensions. Nevertheless, when examining each one of the 22 questions, it became apparent that the 
girls learned to use the game more quickly (Mdn = 5), than the boys ( Mdn = 3), U = 90.5, p = .22, r = .37. 
However, the boys considered the game to be more satisfactory ( Mdn = 4), than the girls ( Mdn = 3), U = 100, p 
= .49, r= .32. 

Lastly, the students were divided into three independent groups, according to their age and grade. Thus, the 
first group consisted of 12 1 st grade students, the second goup of 13 2 nd grade students, and third group of 12 
3 rd grade students. A Kruskal-Wallis test was then applied to evaluate differences among the three grade 
conditions. The results of the analysis indicated a significant effect of grade (x 2 (2, N=37) = 6.74, p - .34) only 
on the students' opinion about the consistency of terms that were used throughout the game. Indeed, the 
follow-up tests that were conducted to evaluate pairwise differences among the three groups revealed that 
the 3 rd grade students did not consider the use of terms to be consistent. In particular, a post-hoc test using 
Mann-Whitney tests with Bonferroni correction showed significant differences between group 1 and 3 (p = .18, 
r = .38), and between group 2 and 3 (p = .42, r = .33). However, there were no significant differences according 
to the students' grade on any other questions or the eight dimensions of the questionnaire. 

6. Discussion and conclusions 

This paper presented the design and development of the prototype of a configurable online 2D game, aimed at 
assisting the educator in the teaching of primary and secondary school Mathematics. Furthermore, the 
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prototype was evaluated through a pilot study and a long-term intervention in real school settings, in order to 
assess its usability aspects and to find any possible flaws. According to the results of the two evaluation 
studies, the students' opinions about the game were mostly positive, and they considered it to be a useful and 
engaging learning tool, regardless of age and gender. Furthermore, concerning the game's usability, most of its 
features elicited average to positive responses from the students and the educators alike. Moreover, the 
educators encountered no difficulties in configuring the game, and the planned educational activities were 
concluded successfully. Thus, it was deduced that the particular game could actually be successfully 
incorporated and used by educators as a supplementary tool for the teaching of formal curricular material. 

These findings are encouraging and suggest that game-based learning activities are well-accepted and 
appreciated by students. Furthermore, the great importance of the educator's feedback and guidance on how 
to use the game became apparent, in accordance with Flays (2005) and Fisch (2005). The work presented in 
this paper had certain limitations; for instance, the game is addressed to younger ages and has a short 
storyline and a limited number of challenges and functions. Additionally, the aforementioned evaluation 
studies focused only on the game's usability, as the researchers' intention was to elicit students' responses to 
this game prototype. Our future work aims at improving and extending the game, by adding new features, 
more hints and help messages, and a longer storyline. Moreover, a multi-player feature should be added to 
the game, in order to assess its impact on student's opinion about the game. Also, it would be worth 
investigating whether cooperative or competitive game playing can enhance students' engagement and 
motivation. Lastly, further research should be conducted in order to examine how the educator's material 
could be more efficiently integrated into the game's storyline and narrative. 

The contribution of this paper is that it described a concrete case study on the creation and evaluation of an 
educational game designed according to instructional game design principles. In more detail, the game 
included interaction and feedback elements, a background story, clear rules, objectives and outcomes, 
combined with educational aspects. As it became apparent through the evaluation process, such a game can 
be easily integrated into the classroom to support the teaching of formal curricular material. Moreover, it can 
be used for distance learning, if certain extensions are considered. Finally, this paper demonstrated that it is 
possible to create a functional prototype of an online game that can be adapted according to the educator's 
specific needs. In conclusion, it is hoped that this study adds to the existing research on instructional games 
and that the presented game will eventually become an effective educational tool. 
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